Hypernetworks for Zero-shot Transfer in Reinforcement Learning

In this paper, hypernetworks are trained to generate behaviors across a range of unseen task conditions, via a novel TD-based training objective and data from a set of near-optimal RL solutions for training tasks. This work relates to meta RL, contextual RL, and transfer learning, with a particular focus on zero-shot performance at test time, enabled by knowledge of the task parameters (also known as context). Our technical approach is based upon viewing each RL algorithm as a mapping from the MDP specifics to the near-optimal value function and policy and seek to approximate it with a hypernetwork that can generate near-optimal value functions and policies, given the parameters of the MDP. We show that, under certain conditions, this mapping can be considered as a supervised learning problem. We empirically evaluate the effectiveness of our method for zero-shot transfer to new reward and transition dynamics on a series of continuous control tasks from DeepMind Control Suite. Our method demonstrates significant improvements over baselines from multitask and meta RL approaches.Comment: AAAI 202

No complexity–stability relationship in empirical ecosystems

International audienceUnderstanding the mechanisms responsible for stability and persistence of ecosystems is one of the greatest challenges in ecology. Robert May showed that, contrary to intuition, complex randomly built ecosystems are less likely to be stable than simpler ones. Few attempts have been tried to test May's prediction empirically, and we still ignore what is the actual complexity–stability relationship in natural ecosystems. Here we perform a stability analysis of 116 quantitative food webs sampled worldwide. We find that classic descriptors of complexity (species richness, connectance and interaction strength) are not associated with stability in empirical food webs. Further analysis reveals that a correlation between the effects of predators on prey and those of prey on predators, combined with a high frequency of weak interactions, stabilize food web dynamics relative to the random expectation. We conclude that empirical food webs have several non-random properties contributing to the absence of a complexity–stability relationship

Effector membrane translocation biosensors reveal G protein and βarrestin coupling profiles of 100 therapeutically relevant GPCRs

The recognition that individual GPCRs can activate multiple signaling pathways has raised the possibility of developing drugs selectively targeting therapeutically relevant ones. This requires tools to determine which G proteins and βarrestins are activated by a given receptor. Here, we present a set of BRET sensors monitoring the activation of the 12 G protein subtypes based on the translocation of their effectors to the plasma membrane (EMTA). Unlike most of the existing detection systems, EMTA does not require modification of receptors or G proteins (except for G(s)). EMTA was found to be suitable for the detection of constitutive activity, inverse agonism, biased signaling and polypharmacology. Profiling of 100 therapeutically relevant human GPCRs resulted in 1500 pathway-specific concentration-response curves and revealed a great diversity of coupling profiles ranging from exquisite selectivity to broad promiscuity. Overall, this work describes unique resources for studying the complexities underlying GPCR signaling and pharmacology